Scaled image generating apparatus and method, image feature calculating apparatus and method, computer programs therefor, and image data structure

ABSTRACT

An apparatus for generating a scaled image divides an image, which comprises a plurality of pixels, into a plurality of blocks by partitioning the image vertically and horizontally, whereby the color space of this image is divided into a plurality of subspaces, referred to as bins (S 10501  to S 10503 ). The apparatus applies a histogram analysis, on a bin-by-bin basis, to the pixels constituting a block of interest among the plurality of blocks. Then, using the result of the histogram analysis, the apparatus decides the representative color of the block of interest in accordance with the average color of pixels belonging to a most frequent bin among the plurality of bins. A scaled image of this image is obtained by applying the above operation to all of the blocks (S 10504  to S 10510 ).

FIELD OF THE INVENTION

[0001] The present invention relates to the field of image processingfor obtaining feature values from an original image (input image) andutilizing these feature values. More particularly, the invention relatesto the field of image processing for scaling an original image.

BACKGROUND OF THE INVENTION

[0002] A variety of image processing is executed in the prior art byobtaining the feature values of an original image and then utilizingthese feature values to execute the processing.

[0003] One example of a method of obtaining such a feature value from anoriginal image involves dividing the original image (the input image)into blocks of vertical and horizontal numbers of pixels that are toundergo scaling, and calculating the mean values of the pixels (pixelvalues) within a plurality of the block images produced by theaforementioned division, thereby generating a scaled image of theoriginal image. The scaled image thus produced is then subjected to thewell-known discrete cosine transform (“DCT” below) and quantizationprocessing, as a result of which coefficients are obtained. Severalcoefficients from the side of low-frequency components are extractedfrom these coefficients as a feature of the original image. The featurethus extracted can be employed as data used in an image search. [SeeISO/IEC JTC1/SC29/WG11/N3522 “MPEG-7 Visual Working Draft 4.0” (VWD 4.0)or ISO/IEC JTC1/SC29/WG11/N3522 “MPEG-7 Visual part of experimentationModel Version 7.0” (VXM 7.0).]

[0004] The conventional procedure for extracting feature values will bedescribed. FIG. 1 is a diagram useful in describing the flow ofprocessing for extracting a color layout descriptor value. Thisprocedure is described in VWD 4.0 or VXM 7.0. FIG. 3 is a flowchartillustrating the processing for extracting a color-layout descriptorvalue.

[0005] In FIGS. 1 and 3, an original image 10001 is scaled to aplurality of blocks of 8×8 pixels each (step S10201). When the scaledimage of the original image is produced, use is made of the mean valueof the pixels within each block obtained by dividing the original imageinto vertical and horizontal numbers of pixels that are the target ofscaling.

[0006] The pixels constituting the generated block images (10011, 10012,10013) of 8×8 pixels each are converted to data (10021, 10022, 10023) inY, Cb, Cr color space (step S10202).

[0007] Next, the data 10021, 10022, 10023 representing the components inY, Cb, Cr color space is subjected to DCT processing (step S10203),whereby DCT coefficients 10031, 10032, 10033 are acquired, and the DCTcoefficients 10031, 10032, 10033 are then subjected to quantization(step S10204).

[0008] In accordance with VWD 4.0 or VXM 7.0, the above-mentioned imagescaling processing, color conversion processing and DCT processing maybe implemented through well-known techniques and the processing is notparticularly standardized.

[0009] Further, with regard to image scaling processing, VWD 4.0 or VXM7.0 merely recommend dividing the original image into blocks of 8×8pixels each and adopting the average color of the pixels within eachblock. For example, in accordance with VWD 4.0, quantization processingfor DC components differs from that for AC components with regard to theY component and Cb/Cr components.

[0010] Next, several coefficients are selected from the side oflow-frequency components among the quantized DCT coefficients (10041,10042, 10043) obtained as a result of quantization processing (stepS10205). In the example of FIG. 1, six coefficients (10051) have beenselected with regard to the coefficients of the Y component, and threecoefficients each (10052, 10053) have been selected with regard to thecoefficients of the Cb/Cr components.

[0011]FIG. 2 is a diagram useful in describing zigzag-scan processingfor selecting coefficients. As exemplified in FIG. 2, the selection ofcoefficients at step S10205 is achieved by rearranging the coefficients,which are arrayed two-dimensionally as indicated by the 8×8 pixelconfiguration, into a one-dimensional array by zigzag scanning, andselecting several coefficients starting from the leading coefficient.The numerals 1 to 64 written in the blocks of FIG. 2 indicate whichnumbers the coefficients will come to occupy starting from the leadingcoefficient after the coefficients have been rearrangedone-dimensionally.

[0012] In accordance with VWD 4.0, the coefficients that should beselected in the coefficient selection process are any of 1, 3, 6, 10,15, 21, 28 and 64. Though the numbers of coefficients are the same forthe Cb-component coefficients and Cr-component coefficients, it ispossible for the number of Y-component coefficients to be set to anumber different from that of the Cb/Cr-component coefficients. With VWD4.0, the default selection is six coefficients with regard to theY-component coefficients and three coefficients for each of theCb/Cr-component coefficients. In other words, VWD 4.0 adopts featuredata (a color layout descriptor value) representing the color layout ofthe original image 1001 using the selected coefficients 10051, 10052,10053.

[0013] If use is made of color layout descriptor values calculated asset forth above with regard to a plurality of images, similar images canbe retrieved. The degree of similarity between items of feature data iscalculated as follows in accordance with VXM 7.0. For example, degree ofsimilarity D between a color layout descriptor value CLD1 (YCoeff,CbCoeff, CrCoeff) and a color layout descriptor value CLD2 (YCoeff′,CbCoeff′, CrCoeff′) of two images is calculated in accordance with thefollowing equation: $\begin{matrix}{D = {\sqrt{\sum\limits_{i = 0}^{{{Max}{\{{{Number}\quad {Of}\quad {YCoeff}}\}}} - 1}{\lambda_{Yi}\left( {{{YCoeff}\lbrack i\rbrack} - {{YCoeff}^{\prime}\lbrack i\rbrack}} \right)}^{2}} + \sqrt{\sum\limits_{i = 0}^{{{Max}{\{{{Number}\quad {Of}\quad {YCCoeff}}\}}} - 1}{\lambda_{Cbi}\left( {{{Cboeff}\lbrack i\rbrack} - {{YCboeff}^{\prime}\lbrack i\rbrack}} \right)}^{2}} + \sqrt{\sum\limits_{i = 0}^{{{Max}{\{{{Number}\quad {Of}\quad {CCoeff}}\}}} - 1}{\lambda_{Cri}\left( {{{CrCoeff}\lbrack i\rbrack} - {{CrCoeff}^{\prime}\lbrack i\rbrack}} \right)}^{2}}}} & (1)\end{matrix}$

[0014] In the above equation, λ indicates weighting relating to eachcoefficient. Weighting values of the kind shown in Table 1 below areindicated in VXM 7.0. The cells in Table 1 that do not show a value haveweighting values of 1. TABLE 1 ORDER OF COEFFICIENTS 1 2 3 4 5 6 Y 2 2 21 1 1 Cb 2 1 1 Cr 4 2 2

[0015] When a scaled image of an original image is produced at stepS10201 in the above-described conventional method of extracting featurevalues, use is made of the mean value of the pixels within a blockobtained by dividing the original image into vertical and horizontalnumbers of pixels that are the target of scaling. As a consequence,information relating to the composition of the original image isunclear.

[0016] Furthermore, several coefficients are extracted from thelow-frequency component side of quantized coefficients obtained byapplying DCT processing to a scaled image, as described above. As aconsequence, the smoothing effect on the original image as a totalsystem becomes too extreme, thereby making the composition informationof the original image even more obscure.

SUMMARY OF THE INVENTION

[0017] The present invention has been proposed to solve the problems ofthe prior art and a first object thereof is to produce a scaled imagethat represents well the features of the image that is to be scaled.

[0018] A second object of the present invention is to calculate afeature value (color layout descriptor) that represents well the colorlayout of an image.

[0019] According to the present invention, the first object is attainedby providing an apparatus for generating a scaled image, comprising:image dividing means for dividing an image, which consists of aplurality of pixels, into a plurality of blocks by partitioning theimage vertically and horizontally; color-space dividing means fordividing color space of the image into a plurality of subspaces; andcolor decision means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks, and deciding a representativecolor of the block of interest in accordance with the average color ofpixels belonging to a most frequent subspace among the plurality ofsubspaces as a result of the histogram calculation.

[0020] If the result of the histogram calculation is that two mostfrequent subspaces exist and that these subspaces are contiguous, thenthe color decision means decides the representative color of the blockof interest in accordance with the average color of pixels belonging tothese two subspaces.

[0021] In a preferred embodiment, the color decision means includes:arithmetic means for calculating a histogram, on a per-subspace basis,with regard to each pixel constituting the block of interest among theplurality of blocks; merge decision means which, if the result of thehistogram calculation is that two most frequent subspaces exist and arenot contiguous or that three or more most frequent subspaces exist, isfor deciding whether a group of other contiguous subspaces can be mergedwith these subspaces to form a unified subspace; subspace merging meanswhich, if the merge decision means has decided that merging is possible,is for merging the group of subspaces, thereby generating the unifiedsubspace; and representative color decision means for recursivelycalculating a most frequent unified subspace by adding up frequencies ofoccurrence that are based upon the histogram calculations with regard tothe unified subspace generated by the subspace merging means, andobtaining the average color of pixels that belong to this most frequentunified subspace and adopting this average color as the representativecolor of the block of interest.

[0022] In another preferred embodiment, the color decision meansincludes arithmetic means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; merge decision means which, ifthe result of the histogram calculation is that two most frequentsubspaces exist and are not contiguous or that three or more mostfrequent subspaces exist, is for deciding whether a group of othercontiguous subspaces can be merged with these subspaces to form aunified subspace; subspace merging means which, if the merge decisionmeans has decided that merging is possible, is for merging the group ofsubspaces, thereby generating the unified subspace; and representativecolor decision means for recursively calculating a most frequent unifiedsubspace by adding up frequencies of occurrence that are based upon thehistogram calculations with regard to the unified subspace generated bythe subspace merging means, and obtaining the average color of pixelsthat belong to this most frequent unified subspace (in an embodimentdescribed below, this is an initial most frequent subspace (bin), orroot subspace, that prevails prior to merging) and adopting this averagecolor as the representative color of the block of interest.

[0023] According to another aspect of the present invention, the firstobject is attained by providing an apparatus for generating a scaledimage, comprising: image dividing means for dividing an image, whichconsists of a plurality of pixels, into a plurality of blocks bypartitioning the image vertically and horizontally; color-space dividingmeans for dividing color space of the image into a plurality ofsubspaces; arithmetic means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; and color decision means which,if the result of the histogram calculation by the arithmetic means isthat a most frequent first subspace among the plurality of subspaces isone only and, moreover, that the difference between pixels belonging tothis first subspace and pixels belonging to a next most frequent secondsubspace after the first subspace is greater than a predetermined firstthreshold value or this difference is greater than a predeterminedsecond threshold value with respect to the overall number of pixelsconstituting the block of interest, is for deciding a representativecolor of the block of interest in accordance with the average color ofpixels belonging to the first subspace.

[0024] According to still another aspect of the present invention, thefirst object is attained by providing an apparatus for generating ascaled image, comprising: image dividing means for dividing an image,which consists of a plurality of pixels, into a plurality of blocks bypartitioning the image vertically and horizontally; color-space dividingmeans for dividing color space of the image into a plurality ofsubspaces; arithmetic means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; and color decision means which,if the result of the histogram calculation by the arithmetic means isthat a most frequent first subspace among the plurality of subspaces isone only and, moreover, that the ratio or proportion between pixelsbelonging to this first subspace and pixels belonging to a next mostfrequent second subspace is greater than a predetermined first thresholdvalue, is for deciding a representative color of the block of interestin accordance with the average color of pixels belonging to the firstsubspace.

[0025] According to the present invention, the second object is attainedby providing an apparatus for calculating features of an image,comprising: the scaled image generating apparatus having any of thestructures described above, and image feature calculation means forcalculating a descriptor, which represents a color-layout feature of ascaled image, based upon the scaled image generated by the scaled imagegenerating apparatus, and retaining at least the descriptor and thescaled image in an associated state.

[0026] In a preferred embodiment, the image feature calculation meansincludes: color-space transformation means for transforming the scaledimage, which has been generated by the scaled image generatingapparatus, to Y, Cb, Cr color space; arithmetic means for applying DCTprocessing to each component information in the Y, Cb, Cr color spaceacquired by the color-space transformation means, and applyingquantization processing to DCT coefficients acquired as a result of theDCT processing; and coefficient selection means for selecting, as adescriptor representing a color-layout feature of the image,coefficients of a prescribed quantity from a low-frequency componentside of the DCT coefficients, quantized for every component in the Y,Cb, Cr color space, acquired from the arithmetic means, and retaining atleast the descriptor and scaled image in associated form.

[0027] Preferably, the apparatus further comprises similaritycalculation means for calculating, based upon descriptors acquired bythe coefficient selection means with regard to two original images,degree of similarity between these original images.

[0028] The foregoing objects are attained also by providing methodscorresponding to the above-described scaled image generating apparatusan image feature calculation apparatus.

[0029] The foregoing objects are attained also by providing computerprograms for instructing a computer to perform an operation that makesit possible to implement the above-described scaled image generatingapparatus an image feature calculation apparatus and the correspondingmethods, as well as a computer-readable storage medium in which thesecomputer programs have been stored.

[0030] Other features and advantages of the present invention will beapparent from the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in andconstitute a part of the specification, illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention.

[0032]FIG. 1 is a diagram useful in describing the flow of processingfor extracting color layout descriptors;

[0033]FIG. 2 is a diagram useful in describing zigzag-scan processingfor selecting coefficients;

[0034]FIG. 3 is a flowchart illustrating processing for extracting colorlayout descriptors according to the prior art;

[0035]FIG. 4 is a diagram exemplifying an original image to beregistered and the image in the divided state;

[0036]FIG. 5 is a conceptual view illustrating the relationship betweenY, Cb, Cr color space and bins;

[0037]FIG. 6 is a flowchart illustrating processing for generating ascaled image in a first embodiment of the present invention;

[0038]FIG. 7 is a flowchart illustrating processing for calculating ablock representative color in the processing for generating a scaledimage according to the first embodiment;

[0039]FIG. 8 is a block diagram illustrating the functional structure ofan image search apparatus according to the first embodiment;

[0040]FIG. 9 is a flowchart illustrating image registration processingexecuted by the image search apparatus of the first embodiment;

[0041]FIG. 10 is a flowchart illustrating image registration processingexecuted by the image search apparatus of the first embodiment;

[0042]FIG. 11 is a flowchart illustrating processing for extractingcolor layout descriptors executed in the image registration processingof the first embodiment;

[0043]FIG. 12 is a diagram exemplifying the scheme of a record within asearch information database according to the first embodiment;

[0044]FIG. 13 is a flowchart illustrating processing for calculating ablock representative color in processing for generating a scaled imageaccording to a second embodiment of the present invention; and

[0045]FIG. 14 is a flowchart illustrating processing for calculating ablock representative color in processing for generating a scaled imageaccording to a third embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] Preferred embodiments of the present invention will now bedescribed in detail in accordance with the accompanying drawings.

[0047] Embodiments of an image search apparatus according to the presentinvention will be described in detail with reference to the drawings.Broadly speaking, in these embodiments, the image search apparatus has afunction for registering images within the apparatus and a function forretrieving images, which are similar to a desired image that is theobject of the search, from among the plurality of images that have beenregistered.

First Embodiment

[0048]FIG. 8 is a block diagram illustrating the functional structure ofthe image search apparatus according to the first embodiment.

[0049] As shown in FIG. 1, the apparatus includes a user interfacemodule 10701 that the operator can switch between image searchprocessing and image registration processing; an image input module10702 for capturing an image (e.g., image data compressed in accordancewith the JPEG standard) via a scanner, digital camera or communicationnetwork; an image memory 10703 for storing an original image temporarilybefore executing calculation of color layout descriptors, describedbelow; and an image storage module 10704 for storing a plurality ofimages, which have been entered via the image input module 10702, in astorage device such as a hard disk or on a portable storage medium.

[0050] The apparatus further includes a color layout descriptorcalculation module 10705 for calculating color layout descriptorsaccording to a characterizing feature of this embodiment; a searchinformation database 10706 for storing calculated color layoutdescriptors, original-image address storage information and attributeinformation; a search condition designating module 10707 that enablesthe operator to designate images and the like to be searched when asearch for similar images is conducted; a similarity calculation module10708 for calculating degree of similarity; and a search-result displaymodule 10709 for displaying search results on a display (not shown).

[0051] In this embodiment, a module indicates a certain function unitthat the image search apparatus is capable of executing, and theembodiment assumes a case where modules are implemented by softwarealone or a case where modules are implemented by hardware and software.

[0052] The image search apparatus having the functional structure shownin FIG. 8 is implementable by a stand-alone information processingdevice such as a personal computer or by a system in which a pluralityof information processing devices operate in concert via variouscommunication lines in the manner of a client-server environment. Thestructure of such an information processing device can be of theordinary type presently available and a detailed description thereof isomitted from the description of this embodiment.

[0053] The image search apparatus of this embodiment having theabove-described functional structure is capable of executing imageregistration processing and image search processing. An overview of thisprocessing will now be described.

Image Registration Processing

[0054] An image (original image) acquired by the image input module10702 is stored in the image storage module 10704 and, in order tocalculate the color layout descriptors(described later), is storedtemporarily in the image memory 10703.

[0055] A color layout descriptor calculated by the color layoutdescriptor calculation module 10705 with regard to the image storedtemporarily in the image memory 10703 is stored in the searchinformation database 10706.

[0056] The color layout descriptor is associated (correlated) with theimage, which has been stored in the image storage module 10704, withimage identification information (referred to as an “image ID” below)issued uniquely in the image search apparatus.

[0057] Thumbnail images, which are for displaying search results, usedin the search-result display module 10709 are stored in the searchinformation database 10706. These thumbnail images are images obtainedby well-known compression in accordance with, e.g., the JPEG (JointPhotographic Experts Group) standard.

Image Search Processing

[0058] With regard to an image of interest (an image that is the objectof a search) that has been selected in the search information database10706 by the operator for the purpose of retrieving similar images, thesimilarity calculation module 10708 calculates degrees of similarity Dbetween this image of interest and all images that have been stored inthe image storage module 10704 in order to detect images that aresimilar to this image.

[0059] A color layout descriptor of the image of interest and colorlayout descriptors that have been stored in the search informationdatabase 10706 are utilized in calculating degree of similarity D. Thesearch-result display module 10709 displays the thumbnail images, whichhave been stored in the search information database 10706, in order ofdecreasing degree of similarity D calculated.

[0060] In addition to the above-mentioned data items, attributeinformation such as keywords and dates of photography may be stored inthe search information database 10706. When an image search is conductedin such case, it will suffice to perform an AND operation between theattribute information and the similar-image based search, perform thesimilarity calculation using color layout descriptors only with regardto images to which the prescribed attribute has been appended, anddisplay the similar images as thumbnail images in order of decreasingdegree of similarity.

[0061] Image registration processing and image search processing willnow be described in greater detail.

Image Registration Processing

[0062]FIG. 9 is a flowchart illustrating image registration processingexecuted by the image search apparatus of the first embodiment. Thisflowchart shows the processing procedure executed by a CPU (not shown)that runs software in which has been coded the operation of thosemodules relating to image registration among the modules depicted inFIG. 8.

[0063] In FIG. 9, an image that has been entered from the image inputmodule 10702 as an image to be registered is stored temporarily in theimage memory 10703, and an image ID is issued for this image (stepS10801). It is required that the issued image ID be managed in such amanner that it will not be a duplicate of other image IDs in the imagesearch apparatus.

[0064] The image for which the image ID has been issued is subjected toprocessing for extracting a color layout descriptor (step S10802). Thedetails of this processing will be described later with reference toFIG. 11.

[0065] The image ID issued at step S10801 and the color layoutdescriptor calculated at step S10802 are stored in the searchinformation database 10706 in associated (correlated) form (stepS10803). Furthermore, this image ID and a thumbnail image (a scaledimage produced at step S10802) of the image to be registered are storedin the search information database 10706 in associated (correlated) form(step S10804).

[0066] The image to be registered that has been stored temporarily inthe image memory 10703 is stored in the image storage module 10704, andthe storage address at which this image file has been stored is storedin the search information database 10706 (step S10805).

[0067] By virtue of the image registration processing described above, arecord is stored in the search information database 10706 with thescheme exemplified in FIG. 12 with regard to the image to be registered.

[0068] It should be noted that the processing of step S10805 describedabove can be executed at any timing so long as it is after the issuanceof the image ID at step S10801. Further, the processing of step S10804also can be executed at any timing so long as it is after the issuanceof the image ID at step S10801. Accordingly, the image registrationprocessing of this embodiment is not limited to the flowchart shown inFIG. 9. However, the order in which the steps S10801 to S10803 areexecuted must not be changed.

Processing for Extracting Color Layout Descriptor

[0069] Processing for extracting a color layout descriptor that is acharacterizing feature of this embodiment will now be described. When animage obtained by scaling the original image to be registered isproduced, this processing for extracting a color layout descriptor iscarried out in order to realize a scaled image that is faithful to thecolor layout of the original image.

[0070] Further, in this embodiment, the encoding of a JPEG-compressedimage or MPEG (Moving Picture coding Experts Group)-1, MPEG-2, MPEG-4frame is carried out in so-called Y, Cb, Cr color space. This is a statein which a color conversion has already been performed at the moment thecode data is decoded.

[0071]FIG. 11 is a flowchart illustrating processing for extracting acolor layout descriptor executed in image registration processingaccording to the first embodiment.

[0072] In the flowchart of FIG. 11, the image (compressed according tothe JPEG standard) to be registered is decoded by a well-knowntechnique. Specifically, the JPEG-compressed image is decomposed intoluminance information and color difference information in Y, Cb, Crcolor space and these items of information are subjected to well-knownDCT processing to thereby perform compressive encoding (step S11001).Accordingly, in this embodiment, information obtained immediatelyfollowing inverse DCT processing is composed of Y, Cb, Cr luminanceinformation and color difference information for each pixel. This is astate in which the Y, Cr, Cr color conversion has already beenimplemented.

[0073] In this embodiment, the thumbnail image of an original image tobe registered is used in displaying the result of an image searchconducted separately. In order that a thumbnail image of an originalimage to be registered may be stored in the search information database10706 in advance, therefore, processing for scaling the image data thatwas acquired at step S11001 is executed (step S11002).

[0074] The scaling processing of step S11002 will now be described indetail.

[0075] With VWD 4.0 or VXM 7.0 described above in connection with theprior art, a scaled image is produced by calculating the average colorof pixels contained in each block into which the original image has beendivided. A problem which arises as a consequence is that informationrelating to the composition of the original image becomes unclear.

[0076] By contrast, in accordance with this embodiment, though theoriginal image is divided into a total of 64 block images (referred tosimply as “blocks” below) by being partitioned into eight blockshorizontally and eight blocks vertically, as shown in FIG. 4, which isthe same as in the prior art, a scaled image of 8×8, or 64, pixels isgenerated by deciding a color that represents each block utilizing amethod, described below, that is a procedure constituting acharacterizing feature of this embodiment.

[0077] However, if a scaled image is generated based upon representativecolors of respective ones of the blocks, as is done in this embodiment,there is the danger that a color that does not exist in the originalimage will appear. This problem becomes particularly conspicuous in acase where colors that are a great distance from each other in colorspace reside in the same block. For example, if a scaled image of theJapanese national flag is produced based upon average color, a pinkcolor will result in relation to a block that straddles both the whitebackground and red circle of the flag. The scaled image thus obtainedwill not be faithful to the color layout of the original image.

[0078] Accordingly, in this embodiment, each block is not represented bythe average color of the pixels belonging to the block. Instead,processing is executed to find a color that represents the block better(i.e., more faithfully).

[0079]FIG. 6 is a flowchart illustrating processing for generating ascaled image according to the first embodiment. This flowchartillustrates the details of the procedure of processing executed at stepS11002 (FIG. 11) for the purpose of realizing a scaled image that isfaithful to the color layout of an original image.

[0080] In FIG. 6, data for dividing Y, Cb, Cr color space into aplurality of subspaces (referred to as “bins” below) is loaded (stepS10501).

[0081]FIG. 5 is a conceptual view showing the relationship between Y,Cb, Cr color space and the bins mentioned above. This roughly decidescolors regarded as identical and is used in order to produce classes forfinding a color histogram.

[0082] Into which bins color space is to be divided should be determinedby experimentation. If there is an approach that has been determined byVWD 4.0 or VXM 7.0, then the division of color space should be performedin accordance therewith.

[0083] Further, the form of the bins within the apparatus may be a LUT(Look-UP table) and mathematical expressions or logical expressions.

[0084] Further, the timing of data loading at step S10501 may be priorto histogram calculation; the timing is not particularly limited.Further, in a case where scaled images of a plurality of images aresought, the data loading at step S10501 need not be performed image byimage; it will suffice to read in the data one time when the initialimage is processed.

[0085] Next, the original image of interest for which the scaled imageis sought is read in (step S10502), then the original image thus read inis divided into a total of 64 blocks by being partitioned into eightblocks vertically and eight blocks horizontally (step S10503), asexemplified in FIG. 4.

[0086] An original image of interest is classified broadly into threetypes, namely portrait (elongated vertically), landscape (elongatedhorizontally) and square. The size of each block should be decided inconformity with the number of blocks into which the original image isdivided. In this embodiment, use is made of a landscape image of Mt.Fuji shown in FIG. 4. If the vertical or horizontal size of the originalimage is such that it cannot be divided evenly by eight, it will sufficeto allocate the remaining pixels to some blocks, and a method ofexceptional treatment based upon this fraction may be employed.

[0087] Next, at steps S10504 to S10510, to which bins of color space thepixels constituting each block apply are determined. Specifically, therepresentative color of each block is decided by histogram calculation.Steps S10504 to S10506 constitute loop processing with respect to acounter I, and steps S10507 to S10509 constitute loop processing withrespect to a counter J. The representative color of each of the 8×8pixel blocks is decided by this loop processing.

[0088] Processing (step S10510) for finding the representative color ofeach block will now be described in detail.

[0089]FIG. 7 is a flowchart illustrating processing for calculating ablock representative color in the processing for generating a scaledimage according to the first embodiment.

[0090] First, at steps S10601 and S10602 in FIG. 7, the bins in Y, Cb,Cr color space to which the pixels of each block obtained by dividingthe original image are to be applied are obtained based upon the resultsof histogram calculation.

[0091] Specifically, with regard to all pixels within a block ofinterest in the present control cycle, the frequency of occurrence ofthese pixels in each bin constituting the Y, Cb, Cr color space, asshown in FIG. 5, and the cumulative value of the pixel values in eachbin are found (step S10601). From the results obtained, the bin forwhich the frequency of occurrence is highest (this bin shall be referredto as the “most frequent bin”) is decided (step S10602).

[0092] If the most frequent bin is determined to be singular (“YES” atstep 10605), then the average color of the pixels belonging to this binis found and this average color is adopted as the representative colorof the block of interest (step S10604).

[0093] On the other hand, if there are two most frequent bins (“YES” atstep S10605), then it is determined whether these two bins arecontiguous in the Y, Cb, Cr color space (step S10606). If the result ofthe determination is that these two bins are contiguous (“YES” at stepS10606), then the average color of the pixels belonging to these twocontiguous bins is found and this average color is adopted as therepresentative color of the block of interest (step S10607).

[0094] Examples of methods of judging whether a plurality of bins aremutually contiguous are a method of storing identification information(ID) of bins that are contiguous to each particular bin, and a method ofassociating geometrical relationships in advance by equations using binIDs. There is no particular limitation upon the method adopted.

[0095] If there are two most frequent bins and these two bins are notcontiguous (“NO” at step S10606), then a group of another plurality ofcontiguous bins is merged with these two bin as a single bin (the binsmerged shall be referred to as a “unified bin” below) and thefrequencies of the merged plurality of bins are added as the frequencyof the unified bin (step 10608).

[0096] To assure that processing will not lapse into an endless loop, adecision is rendered at step S10609 by checking whether bins to bemerged no longer exist. When bin merging has been performed at stepS10608 (“YES” at step S10609), the processing of steps S10603 to S10609is executed recursively, whereby the most frequent unified bin can beobtained. The average color of the pixels belonging to this unified binis found and this is adopted as the representative color of the block ofinterest.

[0097] Furthermore, if three or more most frequent bins exist (“NO” atstep S10605), another group of contiguous bins is merged with thesebins, whereby the frequencies are added (step S10608). The processing ofsteps S10603 to S10609 is executed recursively, whereby the mostfrequent unified bin can be obtained. The average color of the pixelsbelonging to this unified bin is found and this is adopted as therepresentative color of the block of interest.

[0098] The recursive processing mentioned above includes searching for abin contiguous to each most frequent bin and, if this bin and the mostfrequent bin are contiguous, merging these two bins to obtain a singlemost frequency unified bin (step S10608).

[0099] Accordingly, if all of the most frequent bins are contiguous,then the end result is that one unified bin is obtained (“YES” at stepS10603), and therefore the average color of the pixels belonging to theunified bin obtained is found and this average color is adopted as therepresentative color of the block of interest (step S10604).

[0100] On the other hand, if all of the most frequent bins are notcontiguous, then the bin merging processing (step S10608) and theprocessing for obtaining the most frequent unified bin (step S10602) isexecuted recursively until the most frequent unified bin becomes one(“YES” at step S10603) or until two most frequent bins become contiguous(“YES” at step S10606). At step S10604, the average color of pixelsbelonging to the most frequent unified bin obtained is found and thisaverage color is adopted as the representative color of the block ofinterest.

[0101] If the end result is that a most frequent bin is not determined(“NO” at step S10609), the average color of the pixels in the entireblock is found and this average color is adopted as the representativecolor of the block of interest.

[0102] In the above description, when a representative color is decidedin a case where there are two most frequent bins and these two bins arenot contiguous, a unified bin that to serve as the most frequent bin isobtained and the mean value of the pixels belonging to this unified binis adopted as the representative color of the block of interest.However, as the result of experiments conducted by the applicant, it hasbeen found that excellent results are obtained even if the mean value ofpixels belonging to the initial most frequent bin that prevailed priorto unification processing, namely the root bin, is adopted as therepresentative color of the block of interest. This method may also beused to decide the representative color of a block.

[0103] The above-described processing is executed for all 64 of theblocks, thereby making it possible to generate a scaled image thatremains faithful to the color layout of the original image.

[0104] It should be noted that a conceivable method of generating ascaled image is to calculate a histogram in color space obtained bytransformation and then generating the scaled image after colortransformation processing. To cite an example, compressive encoding inthe usual case is carried out after the transformation of color space.In actuality, regardless of whether an image is a JPEG image or a frameimage in an MPEG movie, image compression is such that DCT is performedin Y, Cb, Cr color space. Values in Y, Cb, Cr color space are obtainedby performing inverse DCT in order to effect decoding, and these valuesare converted to values in RGB color space, whereby colors that arenatural to the human eye are displayed.

[0105] With reference again to FIG. 11, the processing of each step upto selection of coefficients is similar to the conventional proceduredescribed above with reference to FIG. 1. Accordingly, this processingwill be described while referring to the reference numerals of FIG. 1.

[0106] Each of the pixels constituting the generated block images(10011, 10012, 10013) of 8×8 pixels each are converted to data (10021,10022, 10023) in Y, Cb, Cr color space at step S10203.

[0107] The data 10021, 10022, 10023 representing the components in Y,Cb, Cr color space is subjected to DCT processing at step S11004,whereby DCT coefficients 10031, 10032, 10033 are obtained.

[0108] The DCT coefficients 10031, 10032, 10033 are subjected toquantization processing at step S11005. In accordance with VWD 4.0, forexample, this quantization processing for DC components differs fromthat for AC components with regard to the Y component and Cb/Crcomponents.

[0109] Next, several coefficients are selected from the side oflow-frequency components among the quantized DCT coefficients 10041,10042, 10043 at step S11006. In the example of FIG. 1, six coefficients(10051) have been selected with regard to the coefficients of the Ycomponent, and three coefficients each (10052, 10053) have been selectedwith regard to the coefficients of the Cb/Cr components.

[0110] As exemplified in FIG. 2, the selection of coefficients at stepS11006 is achieved by rearranging the coefficients, which are arrayedtwo-dimensionally as indicated by the 8×8 pixel configuration, into aone-dimensional array by zigzag scanning, and selecting severalcoefficients starting from the leading coefficient. The numerals 1 to 64written in the blocks of FIG. 2 indicate which numbers the coefficientswill come to occupy starting from the leading coefficient after thecoefficients have been rearranged one-dimensionally. The selectedcoefficients are extracted in order from the side of the low-frequencycomponents. In accordance with VWD 4.0, the number of coefficients thatshould be selected in the coefficient selection process is any of 1, 3,6, 10, 15, 21, 28 and 64.

[0111] Though the numbers of coefficients are the same for theCb-component coefficients and Cr-component coefficients, it is possiblefor the number of Y-component coefficients to be set to a numberdifferent from that of the Cb/Cr-component coefficients. With VWD 4.0,the default selection is six coefficients with regard to the Y-componentcoefficients and three coefficients for each of the Cb/Cr-componentcoefficients. VWD 4.0 adopts a color layout descriptor valuerepresenting the features of color layout of the original image 1001using the selected coefficients 10051, 10052, 10053.

[0112] The color layout descriptor thus calculated is stored in thesearch information database 10706 in a form associated with the imageID, as exemplified in FIG. 12.

[0113]FIG. 12 is a diagram exemplifying the scheme of a record within asearch information database according to the first embodiment.

[0114] As shown in FIG. 12, an image identification number (ID), thestorage address of this image file, the thumbnail image (scaled image)data, the color layout descriptor and other image attribute informationare stored in the search information database 10706 in relation to eachoriginal image. By thus storing these data items in the associatedstate, image search processing (FIG. 10), described below, can beexecuted efficiently. In other words, a convenient and easy-to-use imagesearch can be conducted utilizing color layout descriptors that areindices representing the features of color layout.

[0115] It should be noted that if a thumbnail image is merely to beretrieved and displayed utilizing a color layout descriptor, it willsuffice to adopt a data structure in which at least the thumbnail image(scaled image) of an original image and the color layout descriptor ofthis original image are associated with each other.

[0116] Thus, this embodiment is such that when a scaled image of anoriginal image is generated, a histogram of the pixels within a block ofinterest is calculated with the bins in color space serving as areference, and the color of the block is decided in accordance with themean value of the pixels that belong to the most frequent bin. As aresult, the features of the original image can be expressed efficientlyand a scaled image that is faithful to the color layout of the originalimage can be generated. Furthermore, it is possible to generate a colorlayout descriptor in which the image-composition information of theoriginal image is maintained.

[0117] In the description rendered above, it is explained that when arepresentative color is decided in a case where two most frequent binsexist and these two bins are not contiguous, a unified bin to serve asthe most frequent bin is obtained and the mean value of the pixelsbelonging to the unified bin is adopted as the representative color ofthe block of interest. However, as the result of experiments conductedby the applicant, it has been found that excellent results are obtainedeven if the mean value of pixels belonging to the initial most frequentbin that prevailed prior to unification processing, namely the root bin,is adopted as the representative color of the block of interest.

Image Search Processing

[0118] Image search processing in the image search apparatus of thisembodiment will now be described.

[0119]FIG. 10 is a flowchart illustrating image registration processingexecuted by the image search apparatus according to the firstembodiment. This flowchart shows the processing procedure executed by aCPU (not shown) that runs software in which has been coded the operationof those modules relating to image search among the modules depicted inFIG. 8.

[0120] Original images (referred to also as “registered images” below)are stored in the image storage module 10704 beforehand by theabove-described image registration processing. In image searchprocessing, it is required that color layout descriptors of all of theseregistered image be made available in the search information database10706.

[0121] First, at step S10901 in FIG. 10, the color layout descriptors ofall registered images are read into memory from the search informationdatabase 10706 in accordance with the image search mode selected by theoperator using the user interface module 10701.

[0122] Here it will suffice to adopt an arrangement in which the readingof the color layout descriptors is not carried out whenever an imagesearch is conducted but one time only when the search is first conductedor when the system is started up.

[0123] The image (referred to as the “target image” below) selected bythe operator in the search information database 10706 is set as anoriginal image for the purpose of retrieving similar images (stepS10902). An example of a method of selecting an image at this step is todisplay a plurality of thumbnail images on a display in the form ofrandomly arrayed tiles and then select the desired thumbnail image fromthese thumbnail images. In such a case, it will suffice to generaterandom numbers as image IDs, read the thumbnail images conforming to thecorresponding image IDs out of the search information database 10706 anddisplay these thumbnail images on the display.

[0124] With the image ID that corresponds to the image selected by theoperator serving as a key, the color layout descriptor associated withthis image ID is extracted from the search information database 10706(step S10903).

[0125] The degree of similarity D is calculated in accordance with theabove-cited Equation (1) utilizing the target-image color layoutdescriptor acquired at step S10903 and the color layout descriptors ofall registered image read in at step S10901 (step S10904). For example,if the color layout descriptors of two images are CLD1 (YCoeff, CbCoeff,CrCoeff) and (YCoeff′, CbCoeff′, CrCoeff′), then the degree ofsimilarity D between these two descriptors can be calculated inaccordance with Equation (1) in compliance with the VXM 7.0 standard. InEquation (1), λ indicates weighting relating to each coefficient.Weighting values of the kind shown in Table 1 above are indicated in VXM7.0. The cells in Table 1 that do not show a value have weighting valuesof 1.

[0126] Image IDs are sorted in ascending order (i.e., in order ofincreasing degree of similarity), for example, in accordance with thedegrees of similarity calculated at step S10904 (step S10905), and thethumbnail images corresponding to these sorted image IDs are read infrom the search information database 10706 and displayed on a display(not shown) (step S10906). The reason for sorting in ascending order isas follows: as indicated by Equation (1), the closer the resemblancebetween images, the more the value approaches zero, while the smallerthe resemblance between images, the greater the value becomes.

[0127] In accordance with the image search processing described above,thumbnail images resembling the target image designated by the operatorcan be displayed in list form in order of decreasing degree ofsimilarity. If the operator finds a favorable thumbnail image among thelist thereof, then the operator selects this desired thumbnail image viathe user interface module 10701. Using the image ID of the thumbnailimage selected by the operator as a key, the search-result displaymodule 10709 refers to the search information database 10706 to acquirethe corresponding storage address information, reads the image data,which has been stored in the image storage module 10704, in accordancewith the acquired storage address information and displays the image onthe display.

Second Embodiment

[0128] A second embodiment which has the image search apparatus of thefirst embodiment as its base will now be described. In the descriptionthat follows, components similar to those of the first embodiment willnot be described in order to avoid prolixity, and the description willfocus on the portions constituting the characterizing feature of thisembodiment.

[0129] In this embodiment, processing (step S10510) for obtaining arepresentative color for each block obtained by dividing the originalimage is implemented through a procedure that differs from that of theprocessing (FIG. 7) for calculating block representative colors in theprocessing for generating a scaled image according to the firstembodiment.

[0130]FIG. 13 is a flowchart illustrating processing for calculating ablock representative color in processing for generating a scaled imageaccording to the second embodiment. This processing is applied to eachblock in this embodiment as well.

[0131] First, with regard to all pixels within a block of interest inthe present control cycle, to which bins in Y, Cb, Cr color space thepixels of this block belong are found based upon the result of histogramcalculation (step S11301). That is, with regard to all pixelsconstituting the block of interest, a histogram calculation is performedto obtain the frequency of occurrence of these pixels in each binconstituting the Y, Cb, Cr color space, as shown in FIG. 5, and thecumulative value of the pixel values in each bin.

[0132] Next, by utilizing the results of the histogram calculation, thesystem finds the most frequent bin (the bin for which the frequency ofoccurrence is highest, just as in the case of FIG. 7) and a bin(referred to as the “second bin” below) whose difference in frequencywith respect to the most frequent bin is greater than a predeterminedvalue (step S11302). Here the second bin is the next most frequent binafter the most frequent bin.

[0133] If such a second bin does not exist (“NO” at step S11303) (i.e.,if the most frequent bin is one only and the second bin isnon-existent), then the system obtains the average color of the pixelsbelonging to the most frequent bin found at step S11302 and adopts thisaverage color as the representative color of the block of interest (stepS11304).

[0134] If a “YES” decision is rendered at step S11303, on the otherhand, control proceeds to step S11305. More specifically, if it isjudged at step S11303 that the most frequent bin and second bins exist,it is determined at step S11305 whether the most frequent bin and thesecond bins are all contiguous in Y, Cb, Cr color space.

[0135] If a “YES” decision is rendered at step S11305 (i.e., if the mostfrequent bin and the second bins are contiguous), the system finds theaverage color of the pixels belonging to the second bins and adopts thisaverage color as the representative color of the block of interest (stepS11306).

[0136] On the other hand, if a “NO” decision is rendered at step S11305(i.e., if there are a plurality of most frequent bins or if the mostfrequent bin and the second bins are not all contiguous), then a groupof another plurality of contiguous bins (third bins) contiguous withthese bins is set as unified bin (i.e., as a plurality of bins merged asa single bin, just as in the case of the FIG. 7) and the frequencies ofthese merged plurality of bins are added to give the frequency of theunified bin (step S11307).

[0137] Examples of methods of judging whether a plurality of bins aremutually contiguous are a method of storing identification information(ID) of bins that are contiguous to each particular bin, and a method ofassociating geometrical relationships in advance by equations using binIDs. There is no particular limitation upon the method adopted.

[0138] To assure that processing will not lapse into an endless loop, adecision is rendered at step S11308 by checking whether bins to beunified no longer exist. When bin unification has been performed at stepS11307 (“YES” at step S11308), the processing of steps S11303 to S11308is executed recursively, whereby the most frequent unified bin can beobtained. The average color of the pixels belonging to this unified binis found and this is adopted as the representative color of the block ofinterest.

[0139] The recursive processing mentioned above includes adopting eachmost frequent bin as a reference at step S11305 and searching for a bin(third bin) that is contiguous to these most frequent bins and secondbins.

[0140] If a “YES” decision is rendered at step S11305, a single unifiedbin is eventually obtained and therefore the average color of the pixelsbelonging to the unified bin obtained is found and this average color isadopted as the representative color of the block of interest (stepS11304).

[0141] On the other hand, if a “NO” decision is rendered at step S11305,this means that a group of the aforementioned third bins exists.Accordingly, the unification processing (step S11307) for obtaining asingle unified bin by merging the most frequent bin and third bins andthe processing (step S11302) for obtaining the most frequent unified binis executed recursively until the most frequent bin becomes a single binand a bin whose difference in frequency with respect to the frequency ofthe most frequent unified bin is less than the threshold value no longerexists (“NO” at step S11303) or until the most frequent bin and thirdbins are all contiguous (“YES” at step S11305). At step S11304, theaverage color of pixels belonging to the most frequent unified binobtained is found and this average color is adopted as therepresentative color of the block of interest.

[0142] It should be noted that if, finally, a most frequent bin is notthe sole bin or a bin for which the difference in frequency with respectto the most frequent merged bin is less than the threshold value exists(“NO” at step S11308), the average color of the pixels in the entireblock is found and this average color is adopted as the representativecolor of the block of interest.

[0143] Further, the embodiment described above is such that if a “NO”decision is rendered at step S11305, a unified bin that is the soleunified bin and for which the frequency of occurrence is highest isfound and the mean value of pixels belonging to this unified bin isadopted as the representative color of the block of interest. However,as the result of experiments conducted by the applicant, it has beenfound that excellent results are obtained even if the mean value ofpixels belonging to the initial most frequent bin that prevailed priorto unification processing, namely the root bin, is adopted as therepresentative color of the block of interest. This method may also beused to decide the representative color of a block.

[0144] With regard to the threshold value used in judging the frequencydifference with respect to the most frequent bin in recursiveprocessing, the value may be changed from the threshold value of theinitial cycle of processing. For example, by making this threshold value“1” in recursive processing, it is possible to render a simplesize-comparison judgement.

[0145] Further, the size of the threshold value used in judging thefrequency difference with respect to the most frequent bin may bechanged in accordance with the number of times recursive processing isexecuted. For example, if the threshold value is reduced when the numberof recursive cycles increases, it possible to deal with a case where thefrequency difference does not increase owing to a decrease in binsmerged.

[0146] By applying the above-described processing to all 64 blocks, itis possible to generate a scaled image that remains faithful to thecolor layout of the original image, just as in the first embodiment.

Third Embodiment

[0147] A third embodiment which has the image search apparatus of thefirst and second embodiments as its base will now be described. In thedescription that follows, components similar to those of the firstembodiment will not be described in order to avoid prolixity, and thedescription will focus on the portions constituting the characterizingfeature of this embodiment.

[0148] In this embodiment, processing (step S10510) for obtaining arepresentative color for each block is implemented by still a differentmethod, with the processing procedure described in the second embodimentserving as the basis.

[0149]FIG. 14 is a flowchart illustrating processing for calculating ablock representative color in processing for generating a scaled imageaccording to the third embodiment.

[0150] Unlike the second embodiment (FIG. 13), in which a bin whosedifference in frequency with respect to the most frequent bin is greaterthan a predetermined value is adopted as the “second bin”, the thirdembodiment adopts a bin whose “frequency ratio” (or “frequencyproportion”) with respect to the most frequent bin is greater than apredetermined value is adopted as the “second bin”, as shown in FIG. 14.Since the procedure of this processing is similar to that of FIG. 13, adetailed description thereof is omitted.

[0151] As in the first embodiment, this embodiment also makes itpossible to generate a scaled image that remains faithful to the colorlayout of the original image.

Modifications

[0152] In the embodiments described above, an image acquired by theimage input module 10702 is stored in the image storage module 10704 andis associated with the corresponding color layout descriptor via animage ID stored in the search information database 10706. However, aregistered image need not necessarily be stored in the image storagemodule 10704. For example, the image itself may exist on an Internetserver and the address of the image written in the search informationdatabase 10706 may be the Internet address of the server.

[0153] Further, in the foregoing embodiments, the image searchprocessing is such that a target image is selected and thumbnail imagesthat resemble this image are displayed in order of decreasingsimilarity. However, an embodiment is conceivable in which the targetimage is given as a handwritten sketch, the color layout descriptor ofthe handwritten image is calculated and similar images are retrievedbased upon this color layout descriptor and the color layout descriptorsof the registered images.

[0154] Further, an embodiment is conceivable in which an image that hasnot been registered in the image storage module 10704 is designated, thecolor layout descriptor of this image is calculated and then similarimages are retrieved.

[0155] Further, in the above embodiments, when a list of thumbnailimages is displayed as the results of a similar-image search, thethumbnail images are stored in a field of the image storage module10704. However, there is no limitation upon how these images areretained. The thumbnail images may exist in the form of files and may beassociated with image IDs or with data files of the original images.

[0156] Furthermore, in accordance with scaling processing of an imageaccording to these embodiments, it is possible to create thumbnailimages of excellent quality as viewed by the human eye. Block images arenot limited to those of a fixed size of 8×8 pixels; an original imagemay be divided into blocks of different sizes in conformity with thevertical and horizontal dimensions of the original image, and arepresentative color may be decided, in the manner described above, foreach image block obtained by such division.

Other Embodiments

[0157] The present invention described based upon the foregoingembodiments can be applied to a system constituted by a plurality ofdevices or to an apparatus comprising a single device.

[0158] Furthermore, there are cases where the object of the invention isattained also by supplying a software program, which implements thefunctions of the flowcharts described in each of the foregoingembodiments, directly or remotely to a system or apparatus that operatesas the above-described image search apparatus, reading the suppliedprogram codes with a computer of the system or apparatus, and thenexecuting the program codes. In this case, so long as the system orapparatus has the functions of the program, the mode of implementationneed not rely upon a program.

[0159] Accordingly, since the functions of the present invention areimplemented by computer, the program codes per se installed in thecomputer also implement the present invention. In other words, theclaims of the present invention also cover a computer program that isfor the purpose of implementing the functions of the present invention.

[0160] In this case, so long as the system or apparatus has thefunctions of the program, the form of the program, e.g., object code, aprogram executed by an interpreter or print data supplied to anoperating system, etc., does not matter.

[0161] Examples of storage media that can be used for supplying theprogram are a floppy disk, hard disk, optical disk, magneto-opticaldisk, CD-ROM, CD-R, CD-RW, magnetic tape, non-volatile type memory card,ROM, DVD (DVD-ROM, DVD-R), etc.

[0162] As for the method of supplying the program, the client computercan be connected to a Web page on the Internet using a browser possessedby the client computer, and the computer program per se of the presentinvention or an automatically installable compressed file of the programcan be downloaded to a recording medium such as a hard disk. Further,the program of the present invention can be supplied by dividing theprogram code constituting the program into a plurality of files anddownloading the files from different Web pages. In other words, a WWW(World Wide Web) server that downloads, to multiple users, the programfiles that implement the functions of the present invention by computeralso is covered by the claims of the present invention.

[0163] Further, it is also possible to store the program of the presentinvention on a storage medium such as a CD-ROM upon encrypting theprogram, distribute the storage medium to users, allow users who meetcertain requirements to download decryption key information from a Webpage via the Internet, and allow these users to run the encryptedprogram by using the key information, whereby the program is installedin the user computer.

[0164] Furthermore, besides the case where the aforesaid functionsaccording to the embodiments are implemented by executing the readprogram by computer, an operating system or the like running on thecomputer may perform all or a part of the actual processing so that thefunctions of the foregoing embodiments can be implemented by thisprocessing.

[0165] Furthermore, after the program read from the storage medium iswritten to a function expansion board inserted into the computer or to amemory provided in a function expansion unit connected to the computer,a CPU or the like mounted on the function expansion board or functionexpansion unit performs all or a part of the actual processing so thatthe functions of the foregoing embodiments can be implemented by thisprocessing.

[0166] Thus, in accordance with each of the foregoing embodiments, it ispossible to generate a scaled image that represents well the features ofthe image that is to be scaled

[0167] Further, in accordance with each of the foregoing embodiments, itis possible to calculate a feature value (color layout descriptor) thatrepresents well the color layout of an image.

[0168] As many apparently widely different embodiments of the presentinvention can be made without departing from the spirit and scopethereof, it is to be understood that the invention is not limited to thespecific embodiments thereof except as defined in the appended claims.

What is claimed is:
 1. An apparatus for generating a scaled image,comprising: image dividing means for dividing an image, which consistsof a plurality of pixels, into a plurality of blocks by partitioning theimage vertically and horizontally; color-space dividing means fordividing color space of the image into a plurality of subspaces; andcolor decision means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks, and deciding a representativecolor of the block of interest in accordance with the average color ofpixels belonging to a most frequent subspace among the plurality ofsubspaces as a result of the histogram calculation.
 2. The apparatusaccording to claim 1, wherein if the result of the histogram calculationis that two most frequent subspaces exist and that these subspaces arecontiguous, then said color decision means decides the representativecolor of the block of interest in accordance with the average color ofpixels belonging to these two subspaces.
 3. The apparatus according toclaim 1, wherein said color decision means includes: arithmetic meansfor performing a histogram calculation, on a per-subspace basis, withregard to pixels constituting a block of interest among the plurality ofblocks; merge decision means which, if the result of the histogramcalculation is that two most frequent subspaces exist and are notcontiguous or that three or more most frequent subspaces exist, is fordeciding whether a group of other contiguous subspaces can be mergedwith these subspaces to form a unified subspace; subspace merging meanswhich, if said merge decision means has decided that merging ispossible, is for merging the group of subspaces, thereby generating theunified subspace; and representative color decision means forrecursively calculating a most frequent unified subspace by adding upfrequencies of occurrence that are based upon the histogram calculationswith regard to the unified subspace generated by said subspace mergingmeans, and obtaining the average color of pixels that belong to thismost frequent unified subspace and adopting this average color as therepresentative color of the block of interest.
 4. The apparatusaccording to claim 1, wherein said color decision means includes:arithmetic means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; merge decision means which, ifthe result of the histogram calculation is that two most frequentsubspaces exist and are not contiguous or that three or more mostfrequent subspaces exist, is for deciding whether a group of othercontiguous subspaces can be merged with these subspaces to form aunified subspace; subspace merging means which, if said merge decisionmeans has decided that merging is possible, is for merging the group ofsubspaces, thereby generating the unified subspace; and representativecolor decision means for recursively calculating a most frequent unifiedsubspace by adding up frequencies of occurrence that are based upon thehistogram calculations with regard to the unified subspace generated bysaid subspace merging means, and obtaining the average color of pixelsthat belong to this most frequent unified subspace and adopting thisaverage color as the representative color of the block of interest. 5.An apparatus for generating a scaled image, comprising: image dividingmeans for dividing an image, which consists of a plurality of pixels,into a plurality of blocks by partitioning the image vertically andhorizontally; color-space dividing means for dividing color space of theimage into a plurality of subspaces; arithmetic means for performing ahistogram calculation, on a per-subspace basis, with regard to pixelsconstituting a block of interest among the plurality of blocks; andcolor decision means which, if the result of the histogram calculationby said arithmetic means is that a most frequent first subspace amongthe plurality of subspaces is one only and, moreover, that thedifference between pixels belonging to this first subspace and pixelsbelonging to a next most frequent second subspace after the firstsubspace is greater than a predetermined first threshold value or thisdifference is greater than a predetermined second threshold value withrespect to the overall number of pixels constituting the block ofinterest, is for deciding a representative color of the block ofinterest in accordance with the average color of pixels belonging to thefirst subspace.
 6. The apparatus according to claim 5, wherein if theresult of the histogram calculation is that the first subspace is oneonly and, moreover, that all third subspaces for which the difference inpixels with respect to the pixels belonging to the first subspace isless than a predetermined third threshold value are contiguous to thefirst subspace, said color decision means decides a representative colorof the block of interest in accordance with the average color of pixelsbelonging to the third subspaces.
 7. The apparatus according to claim 5,wherein said color decision means includes: merge decision means which,if the result of the histogram calculation is that a single or pluralityof fourth subspaces for which the difference in pixels with respect tothe pixels belonging to the first subspace is less than a predeterminedfourth threshold value exist, is for deciding whether a group of othercontiguous subspaces can be merged with these subspaces to form aunified subspace; subspace merging means for generating a single unifiedsubspace comprising the first and fourth subspaces and the group ofother subspaces if said merge decision means has decided that merging ispossible; and representative color decision means for adding upfrequencies of occurrence that are based upon the histogram calculationswith regard to the unified subspace generated by said subspace mergingmeans, thereby calculating a most frequent unified subspace thatsatisfies the requirement that the first subspace be one only and thatthe difference between pixels belonging to a most frequent first unifiedsubspace and pixels belonging to a next most frequent second unifiedsubspace be greater than a fifth threshold value that is fixed or thatvaries in dependence upon number of recursive processing cycles, andobtaining the average color of pixels that belong to this unifiedsubspace and adopting this average color as the representative color ofthe block of interest.
 8. The apparatus according to claim 5, whereinsaid color decision means includes: merge decision means which, if theresult of the histogram calculation is that two of the first subspacesexist and are not contiguous or that three or more of the firstsubspaces exist, is for deciding whether a group of other contiguoussubspaces can be merged with these subspaces to form a unified subspace;subspace merging means for merging the group of subspaces to therebygenerate a unified subspace if said merge decision means has decidedthat merging is possible; first representative color means for adding upfrequencies of occurrence that are based upon the histogram calculationswith regard to the unified subspace generated by said subspace mergingmeans, thereby calculating a most frequent unified subspace thatsatisfies the requirement that the first subspace be one only and thatthe difference between pixels belonging to a most frequent first unifiedsubspace and pixels belonging to a next most frequent second unifiedsubspace be less than a sixth threshold value that is fixed or thatvaries in dependence upon number of recursive processing cycles; and ifthe first subspace that prevails prior to merging and that is thestarting point of this unified subspace is a single subspace, obtainingthe average color of pixels that belong to this first subspace andadopting this average color as the representative color of the block ofinterest; and second representative color decision means which, if thefirst subspace that prevails prior to merging and that is said startingpoint is plural in number, is for obtaining the average color of pixelsthat belong to all of these first subspaces, obtaining the average colorof pixels that belong to the unified subspace, and adopting the averagecolor of these two average colors as the representative color of theblock of interest.
 9. An apparatus for generating a scaled image,comprising: image dividing means for dividing an image, which consistsof a plurality of pixels, into a plurality of blocks by partitioning theimage vertically and horizontally; color-space dividing means fordividing color space of the image into a plurality of subspaces;arithmetic means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; and color decision means which,if the result of the histogram calculation by said arithmetic means isthat a most frequent first subspace among the plurality of subspaces isone only and, moreover, that the ratio or proportion between pixelsbelonging to this first subspace and pixels belonging to a next mostfrequent second subspace is greater than a predetermined first thresholdvalue, is for deciding a representative color of the block of interestin accordance with the average color of pixels belonging to the firstsubspace.
 10. The apparatus according to claim 9, wherein if the resultof the histogram calculation is that the first subspace is one only and,moreover, that all third subspaces for which the ratio or proportion ofpixels with respect to the pixels belonging to the first subspace isless than a predetermined second threshold value are contiguous to thefirst subspace, said color decision means decides a representative colorof the block of interest in accordance with the average color of pixelsbelonging to a fourth subspace for which the ratio or proportion ofpixels with respect to the pixels belonging to the first subspace isgreater than a predetermined third threshold value.
 11. The apparatusaccording to claim 9, wherein said color decision means includes: mergedecision means which, if the result of the histogram calculation is thata single or plurality of fifth subspaces for which the ratio orproportion of pixels with respect to the pixels belonging to the firstsubspace is less than a predetermined fifth threshold value exist, isfor deciding whether a group of other contiguous subspaces can be mergedwith these subspaces to form a unified subspace; subspace merging meansfor generating a single unified subspace comprising the first and fourthsubspaces and the group of other subspaces if said merge decision meanshas decided that merging is possible; and representative color decisionmeans for adding up frequencies of occurrence that are based upon thehistogram calculations with regard to the unified subspace generated bysaid subspace merging means, thereby calculating a most frequent unifiedsubspace that satisfies the requirement that the first subspace be oneonly and that the ratio or proportion between pixels belonging to a mostfrequent first unified subspace and pixels belonging to a next mostfrequent second unified subspace be greater than a sixth threshold valuethat is fixed or that varies in dependence upon number of recursiveprocessing cycles, and obtaining the average color of pixels that belongto this unified subspace and adopting this average color as therepresentative color of the block of interest.
 12. The apparatusaccording to claim 9, wherein said color decision means includes: mergedecision means which, if the result of the histogram calculation is thata single or plurality of six subspaces for which the ratio or proportionof pixels with respect to the pixels belonging to the first subspace isgreater than a predetermined seventh threshold value exist, is fordeciding whether a group of other contiguous subspaces can be mergedwith these subspaces to form a unified subspace; subspace merging meansfor generating a single unified subspace comprising the first and sixthsubspaces and the group of other subspaces if said merge decision meanshas decided that merging is possible; first representative color meansfor adding up frequencies of occurrence that are based upon thehistogram calculations with regard to the unified subspace generated bysaid subspace merging means, thereby calculating a most frequent unifiedsubspace that satisfies the requirement that the first subspace be oneonly and that the ratio or proportion between pixels belonging to a mostfrequent first unified subspace and pixels belonging to a next mostfrequent second unified subspace be less than a seventh threshold valuethat is fixed or that varies in dependence upon number of recursiveprocessing cycles; and if the first subspace that prevails prior tomerging and that is the starting point of this unified subspace is asingle subspace, obtaining the average color of pixels that belong tothis first subspace and adopting this average color as therepresentative color of the block of interest; and second representativecolor decision means which, if the first subspace that prevails prior tomerging and that is said starting point is plural in number, is forobtaining the average color of pixels that belong to all of these firstsubspaces, obtaining the average color of pixels that belong to theunified subspace, and adopting the average color of these two averagecolors as the representative color of the block of interest.
 13. Anapparatus for calculating features of an image, comprising: the scaledimage generating apparatus set forth in claim 1; and image featurecalculation means for calculating a descriptor, which represents acolor-layout feature of a scaled image, based upon the scaled imagegenerated by said scaled image generating apparatus, and retaining atleast the descriptor and the scaled image in an associated state. 14.The apparatus according to claim 13, wherein said image featurecalculation means includes: color-space transformation means fortransforming the scaled image, which has been generated by said scaledimage generating apparatus, to Y, Cb, Cr color space; arithmetic meansfor applying DCT processing to each component information in the Y, Cb,Cr color space acquired by said color-space transformation means, andapplying quantization processing to DCT coefficients acquired as aresult of the DCT processing; and coefficient selection means forselecting, as a descriptor representing a color-layout feature of theimage, coefficients of a prescribed quantity from a low-frequencycomponent side of the DCT coefficients, quantized for every component inthe Y, Cb, Cr color space, acquired from said arithmetic means, andretaining at least the descriptor and scaled image in associated form.15. The apparatus according to claim 14, further comprising similaritycalculation means for calculating, based upon descriptors acquired bysaid coefficient selection means with regard to two original images,degree of similarity between these original images.
 16. The apparatusaccording to claim 15, wherein said similarity calculation means: sumsvalues obtained by weighting squares of differences betweencorresponding coefficients that have been selected as the descriptorswith regard to respective ones of the two original images, wherein theweighting applied depends upon the positions of the coefficients; takessquare roots of the sums obtained, thereby acquiring the Y, Cb, Crcomponents; adds up values corresponding to these Y, Cb, Cr componentsto thereby obtain a sum; and adopts this sum as distance between the twooriginal images.
 17. An image data structure in which at least a scaledimage of an original image and a descriptor representing a color-layoutfeature of this original image are associated with each other.
 18. Amethod of generating a scaled image, comprising: an image dividing stepof dividing an image, which consists of a plurality of pixels, into aplurality of blocks by partitioning the image vertically andhorizontally; a color-space dividing step of dividing color space of theimage into a plurality of subspaces; and a color decision step ofperforming a histogram calculation, on a per-subspace basis, with regardto pixels constituting a block of interest among the plurality ofblocks, and deciding a representative color of the block of interest inaccordance with the average color of pixels belonging to a most frequentsubspace among the plurality of subspaces as a result of the histogramcalculation.
 19. The method according to claim 18, wherein if the resultof the histogram calculation is that two most frequent subspaces existand that these subspaces are contiguous, then said color decision stepdecides the representative color of the block of interest in accordancewith the average color of pixels belonging to these two subspaces. 20.The method according to claim 18, wherein said color decision stepincludes: an arithmetic step of performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; if the result of the histogramcalculation is that two most frequent subspaces exist and are notcontiguous or that three or more most frequent subspaces exist, a mergedecision step of deciding whether a group of other contiguous subspacescan be merged with these subspaces to form a unified subspace; asubspace merging step of merging the group of subspaces, therebygenerating the unified subspace, if it is decided at said merge decisionstep that merging is possible; and a representative color decision stepof recursively calculating a most frequent unified subspace by adding upfrequencies of occurrence that are based upon the histogram calculationswith regard to the unified subspace generated at said subspace mergingstep, and obtaining the average color of pixels that belong to this mostfrequent unified subspace and adopting this average color as therepresentative color of the block of interest.
 21. The method accordingto claim 18, wherein said color decision step includes: an arithmeticstep of performing a histogram calculation, on a per-subspace basis,with regard to pixels constituting a block of interest among theplurality of blocks; if the result of the histogram calculation is thattwo most frequent subspaces exist and are not contiguous or that threeor more most frequent subspaces exist, a merge decision step of decidingwhether a group of other contiguous subspaces can be merged with thesesubspaces to form a unified subspace; a subspace merging step of mergingthe group of subspaces, thereby generating the unified subspace, if itis decided at said merge decision step that merging is possible; and arepresentative color decision step of recursively calculating a mostfrequent unified subspace by adding up frequencies of occurrence thatare based upon the histogram calculations with regard to the unifiedsubspace generated at said subspace merging step, and obtaining theaverage color of pixels that belong to this most frequent unifiedsubspace and adopting this average color as the representative color ofthe block of interest.
 22. A method of generating a scaled image,comprising: an image dividing step of dividing an image, which consistsof a plurality of pixels, into a plurality of blocks by partitioning theimage vertically and horizontally; a color-space dividing step ofdividing color space of the image into a plurality of subspaces; and anarithmetic step of performing a histogram calculation, on a per-subspacebasis, with regard to pixels constituting a block of interest among theplurality of blocks; and if the result of the histogram calculation atsaid arithmetic step is that a most frequent first subspace among theplurality of subspaces is one only and, moreover, that the differencebetween pixels belonging to this first subspace and pixels belonging toa next most frequent second subspace after the first subspace is greaterthan a predetermined first threshold value or this difference is greaterthan a predetermined second threshold value with respect to the overallnumber of pixels constituting the block of interest, a color decisionstep of deciding a representative color of the block of interest inaccordance with the average color of pixels belonging to the firstsubspace.
 23. The method according to claim 22, wherein if the result ofthe histogram calculation is that the first subspace is one only and,moreover, that all third subspaces for which the difference in pixelswith respect to the pixels belonging to the first subspace is less thana predetermined third threshold value are contiguous to the firstsubspace, said color decision step decides a representative color of theblock of interest in accordance with the average color of pixelsbelonging to the third subspaces.
 24. The method according to claim 22,wherein said color decision step includes: if the result of thehistogram calculation is that a single or plurality of fourth subspacesfor which the difference in pixels with respect to the pixels belongingto the first subspace is less than a predetermined fourth thresholdvalue exist, a merge decision step of deciding whether a group of othercontiguous subspaces can be merged with these subspaces to form aunified subspace; a subspace merging step of generating a single unifiedsubspace comprising the first and fourth subspaces and the group ofother subspaces if it has been decided at said merge decision step thatmerging is possible; and a representative color decision step of addingup frequencies of occurrence that are based upon the histogramcalculations with regard to the unified subspace generated at saidsubspace merging step, thereby calculating a most frequent unifiedsubspace that satisfies the requirement that the first subspace be oneonly and that the difference between pixels belonging to a most frequentfirst unified subspace and pixels belonging to a next most frequentsecond unified subspace be greater than a fifth threshold value that isfixed or that varies in dependence upon number of recursive processingcycles, and obtaining the average color of pixels that belong to thisunified subspace and adopting this average color as the representativecolor of the block of interest.
 25. The method according to claim 22,wherein said color decision step includes: if the result of thehistogram calculation is that two of the first subspaces exist and arenot contiguous or that three or more of the first subspaces exist, amerge decision step of deciding whether a group of other contiguoussubspaces can be merged with these subspaces to form a unified subspace;a subspace merging step of merging the group of subspaces to therebygenerate a unified subspace if it has been decided at said mergedecision step that merging is possible; and a first representative colorstep of adding up frequencies of occurrence that are based upon thehistogram calculations with regard to the unified subspace generated atsaid subspace merging step, thereby calculating a most frequent unifiedsubspace that satisfies the requirement that the first subspace be oneonly and that the difference between pixels belonging to a most frequentfirst unified subspace and pixels belonging to a next most frequentsecond unified subspace be less than a sixth threshold value that isfixed or that varies in dependence upon number of recursive processingcycles; and if the first subspace that prevails prior to merging andthat is the starting point of this unified subspace is a singlesubspace, obtaining the average color of pixels that belong to thisfirst subspace and adopting this average color as the representativecolor of the block of interest; and if the first subspace that prevailsprior to merging and that is said starting point is plural in number, asecond representative color decision step of obtaining the average colorof pixels that belong to all of these first subspaces, obtaining theaverage color of pixels that belong to the unified subspace, andadopting the average color of these two average colors as therepresentative color of the block of interest.
 26. A method ofgenerating a scaled image, comprising: an image dividing step ofdividing an image, which consists of a plurality of pixels, into aplurality of blocks by partitioning the image vertically andhorizontally; a color-space dividing step of dividing color space of theimage into a plurality of subspaces; and an arithmetic step ofperforming a histogram calculation, on a per-subspace basis, with regardto pixels constituting a block of interest among the plurality ofblocks; and if the result of the histogram calculation at saidarithmetic step is that a most frequent first subspace among theplurality of subspaces is one only and, moreover, that the ratio orproportion between pixels belonging to this first subspace and pixelsbelonging to a next most frequent second subspace is greater than apredetermined first threshold value, a color decision step of deciding arepresentative color of the block of interest in accordance with theaverage color of pixels belonging to the first subspace.
 27. The methodaccording to claim 26, wherein if the result of the histogramcalculation is that the first subspace is one only and, moreover, thatall third subspaces for which the ratio or proportion of pixels withrespect to the pixels belonging to the first subspace is less than apredetermined second threshold value are contiguous to the firstsubspace, said color decision step decides a representative color of theblock of interest in accordance with the average color of pixelsbelonging to a fourth subspace for which the ratio or proportion ofpixels with respect to the pixels belonging to the first subspace isgreater than a predetermined third threshold value.
 28. The methodaccording to claim 26, wherein said color decision step includes: if theresult of the histogram calculation is that a single or plurality offifth subspaces for which the ratio or proportion of pixels with respectto the pixels belonging to the first subspace is less than apredetermined fifth threshold value exist, a merge decision step ofdeciding whether a group of other contiguous subspaces can be mergedwith these subspaces to form a unified subspace; a subspace merging stepof generating a single unified subspace comprising the first and fourthsubspaces and the group of other subspaces if it has been decided atsaid merge decision step that merging is possible; and a representativecolor decision step of adding up frequencies of occurrence that arebased upon the histogram calculations with regard to the unifiedsubspace generated at said subspace merging step, thereby calculating amost frequent unified subspace that satisfies the requirement that thefirst subspace be one only and that the ratio or proportion betweenpixels belonging to a most frequent first unified subspace and pixelsbelonging to a next most frequent second unified subspace be greaterthan a sixth threshold value that is fixed or that varies in dependenceupon number of recursive processing cycles, and obtaining the averagecolor of pixels that belong to this unified subspace and adopting thisaverage color as the representative color of the block of interest. 29.The method according to claim 26, wherein said color decision stepincludes: if the result of the histogram calculation is that a single orplurality of six subspaces for which the ratio or proportion of pixelswith respect to the pixels belonging to the first subspace is greaterthan a predetermined seventh threshold value exist, a merge decisionstep of deciding whether a group of other contiguous subspaces can bemerged with these subspaces to form a unified subspace; a subspacemerging step of generating a single unified subspace comprising thefirst and sixth subspaces and the group of other subspaces if it hasbeen decided at said merge decision step that merging is possible; afirst representative color step of adding up frequencies of occurrencethat are based upon the histogram calculations with regard to theunified subspace generated at said subspace merging step, therebycalculating a most frequent unified subspace that satisfies therequirement that the first subspace be one only and that the ratio orproportion between pixels belonging to a most frequent first unifiedsubspace and pixels belonging to a next most frequent second unifiedsubspace be less than a seventh threshold value that is fixed or thatvaries in dependence upon number of recursive processing cycles; and ifthe first subspace that prevails prior to merging and that is thestarting point of this unified subspace is a single subspace, obtainingthe average color of pixels that belong to this first subspace andadopting this average color as the representative color of the block ofinterest; and if the first subspace that prevails prior to merging andthat is said starting point is plural in number, a second representativecolor decision step of obtaining the average color of pixels that belongto all of these first subspaces, obtaining the average color of pixelsthat belong to the unified subspace, and adopting the average color ofthese two average colors as the representative color of the block ofinterest.
 30. An method for calculating features of an image,comprising: the scaled image generating method set forth in claim 18;and an image feature calculation step of calculating a descriptor, whichrepresents a color-layout feature of a scaled image, based upon thescaled image generated by said scaled image generating method, andretaining at least the descriptor and the scaled image in an associatedstate.
 31. The method according to claim 30, wherein said image featurecalculation step includes: a color-space transformation step oftransforming the scaled image, which has been generated by said scaledimage generating method, to Y, Cb, Cr color space; an arithmetic step ofapplying DCT processing to each component information in the Y, Cb, Crcolor space acquired at said color-space transformation step, andapplying quantization processing to DCT coefficients acquired as aresult of the DCT processing; and a coefficient selection step ofselecting, as a descriptor representing a color-layout feature of theimage, coefficients of a prescribed quantity from a low-frequencycomponent side of the DCT coefficients, quantized for every component inthe Y, Cb, Cr color space, acquired at said arithmetic step, andretaining at least the descriptor and scaled image in associated form.32. An apparatus for calculating features of an image, comprising: thescaled image generating apparatus set forth in claim 5; and imagefeature calculation means for calculating a descriptor, which representsa color-layout feature of a scaled image, based upon the scaled imagegenerated by said scaled image generating apparatus, and retaining atleast the descriptor and the scaled image in an associated state. 33.The apparatus according to claim 32, wherein said image featurecalculation means includes: color-space transformation means fortransforming the scaled image, which has been generated by said scaledimage generating apparatus, to Y, Cb, Cr color space; arithmetic meansfor applying DCT processing to each component information in the Y, Cb,Cr color space acquired by said color-space transformation means, andapplying quantization processing to DCT coefficients acquired as aresult of the DCT processing; and coefficient selection means forselecting, as a descriptor representing a color-layout feature of theimage, coefficients of a prescribed quantity from a low-frequencycomponent side of the DCT coefficients, quantized for every component inthe Y, Cb, Cr color space, acquired from said arithmetic means, andretaining at least the descriptor and scaled image in associated form.34. The apparatus according to claim 33, further comprising similaritycalculation means for calculating, based upon descriptors acquired bysaid coefficient selection means with regard to two original images,degree of similarity between these original images.
 35. The apparatusaccording to claim 34, wherein said similarity calculation means: sumsvalues obtained by weighting squares of differences betweencorresponding coefficients that have been selected as the descriptorswith regard to respective ones of the two original images, wherein theweighting applied depends upon the positions of the coefficients; takessquare roots of the sums obtained, thereby acquiring the Y, Cb, Crcomponents; adds up values corresponding to these Y, Cb, Cr componentsto thereby obtain a sum; and adopts this sum as distance between the twooriginal images.
 36. An apparatus for calculating features of an image,comprising: the scaled image generating apparatus set forth in claim 5;and image feature calculation means for calculating a descriptor, whichrepresents a color-layout feature of a scaled image, based upon thescaled image generated by said scaled image generating apparatus, andretaining at least the descriptor and the scaled image in an associatedstate.
 37. The apparatus according to claim 36, wherein said imagefeature calculation means includes: color-space transformation means fortransforming the scaled image, which has been generated by said scaledimage generating apparatus, to Y, Cb, Cr color space; arithmetic meansfor applying DCT processing to each component information in the Y, Cb,Cr color space acquired by said color-space transformation means, andapplying quantization processing to DCT coefficients acquired as aresult of the DCT processing; and coefficient selection means forselecting, as a descriptor representing a color-layout feature of theimage, coefficients of a prescribed quantity from a low-frequencycomponent side of the DCT coefficients, quantized for every component inthe Y, Cb, Cr color space, acquired from said arithmetic means, andretaining at least the descriptor and scaled image in associated form.38. The apparatus according to claim 37, further comprising similaritycalculation means for calculating, based upon descriptors acquired bysaid coefficient selection means with regard to two original images,degree of similarity between these original images.
 39. The apparatusaccording to claim 38, wherein said similarity calculation means: sumsvalues obtained by weighting squares of differences betweencorresponding coefficients that have been selected as the descriptorswith regard to respective ones of the two original images, wherein theweighting applied depends upon the positions of the coefficients; takessquare roots of the sums obtained, thereby acquiring the Y, Cb, Crcomponents; adds up values corresponding to these Y, Cb, Cr componentsto thereby obtain a sum; and adopts this sum as distance between the twooriginal images.
 40. A method of calculating features of an image,comprising: the scaled image generating method set forth in claim 22;and an image feature calculation step of calculating a descriptor, whichrepresents a color-layout feature of a scaled image, based upon thescaled image generated by said scaled image generating method, andretaining at least the descriptor and the scaled image in an associatedstate.
 41. The method according to claim 40, wherein said image featurecalculation step includes: a color-space transformation step oftransforming the scaled image, which has been generated by said scaledimage generating method, to Y, Cb, Cr color space; an arithmetic step ofapplying DCT processing to each component information in the Y, Cb, Crcolor space acquired at said color-space transformation step, andapplying quantization processing to DCT coefficients acquired as aresult of the DCT processing; and a coefficient selection step ofselecting, as a descriptor representing a color-layout feature of theimage, coefficients of a prescribed quantity from a low-frequencycomponent side of the DCT coefficients, quantized for every component inthe Y, Cb, Cr color space, acquired at said arithmetic step, andretaining at least the descriptor and scaled image in associated form.42. A method of calculating features of an image, comprising: the scaledimage generating method set forth in claim 26; and an image featurecalculation step of calculating a descriptor, which represents acolor-layout feature of a scaled image, based upon the scaled imagegenerated by said scaled image generating method, and retaining at leastthe descriptor and the scaled image in an associated state.
 43. Themethod according to claim 42, wherein said image feature calculationstep includes: a color-space transformation step of transforming thescaled image, which has been generated by said scaled image generatingmethod, to Y, Cb, Cr color space; an arithmetic step of applying DCTprocessing to each component information in the Y, Cb, Cr color spaceacquired at said color-space transformation step, and applyingquantization processing to DCT coefficients acquired as a result of theDCT processing; and a coefficient selection step of selecting, as adescriptor representing a color-layout feature of the image,coefficients of a prescribed quantity from a low-frequency componentside of the DCT coefficients, quantized for every component in the Y,Cb, Cr color space, acquired at said arithmetic step, and retaining atleast the descriptor and scaled image in associated form.
 44. A computerprogram comprising operating commands for instructing a computer tooperate as a scaled image generating apparatus, which comprises: imagedividing means for dividing an image, which consists of a plurality ofpixels, into a plurality of blocks by partitioning the image verticallyand horizontally; color-space dividing means for dividing color space ofthe image into a plurality of subspaces; and color decision means forperforming a histogram calculation, on a per-subspace basis, with regardto pixels constituting a block of interest among the plurality ofblocks, and deciding a representative color of the block of interest inaccordance with the average color of pixels belonging to a most frequentsubspace among the plurality of subspaces as a result of the histogramcalculation.
 45. A computer program comprising operating commands forinstructing a computer to operate as a scaled image generatingapparatus, which comprises: image dividing means for dividing an image,which consists of a plurality of pixels, into a plurality of blocks bypartitioning the image vertically and horizontally; color-space dividingmeans for dividing color space of the image into a plurality ofsubspaces; arithmetic means for performing a histogram calculation, on aper-subspace basis, with regard to pixels constituting a block ofinterest among the plurality of blocks; and color decision means which,if the result of the histogram calculation by said arithmetic means isthat a most frequent first subspace among the plurality of subspaces isone only and, moreover, that the difference between pixels belonging tothis first subspace and pixels belonging to a next most frequent secondsubspace after the first subspace is greater than a predetermined firstthreshold value or this difference is greater than a predeterminedsecond threshold value with respect to the overall number of pixelsconstituting the block of interest, is for deciding a representativecolor of the block of interest in accordance with the average color ofpixels belonging to the first subspace.
 46. A computer programcomprising operating commands for instructing a computer to operate as ascaled image generating apparatus, which comprises: image dividing meansfor dividing an image, which consists of a plurality of pixels, into aplurality of blocks by partitioning the image vertically andhorizontally; color-space dividing means for dividing color space of theimage into a plurality of subspaces; and arithmetic means for performinga histogram calculation, on a per-subspace basis, with regard to pixelsconstituting a block of interest among the plurality of blocks; andcolor decision means which, if the result of the histogram calculationby said arithmetic means is that a most frequent first subspace amongthe plurality of subspaces is one only and, moreover, that the ratio orproportion between pixels belonging to this first subspace and pixelsbelonging to a next most frequent second subspace is greater than apredetermined first threshold value, is for deciding a representativecolor of the block of interest in accordance with the average color ofpixels belonging to the first subspace.
 47. A computer program set forthin claim 44, said computer program further comprising operating commandsfor calculating a descriptor, which represents a color-layout feature ofa scaled image, based upon the scaled image generated by the computerthat operates as said scaled image generating apparatus, and retainingat least the descriptor and the scaled image in an associated state. 48.A computer program set forth in claim 45, said computer program furthercomprising operating commands for calculating a descriptor, whichrepresents a color-layout feature of a scaled image, based upon thescaled image generated by the computer that operates as said scaledimage generating apparatus, and retaining at least the descriptor andthe scaled image in an associated state.
 49. A computer program setforth in claim 46, said computer program further comprising operatingcommands for calculating a descriptor, which represents a color-layoutfeature of a scaled image, based upon the scaled image generated by thecomputer that operates as said scaled image generating apparatus, andretaining at least the descriptor and the scaled image in an associatedstate.